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Abstract 

In  this  paper,  we  derive  statistical  selection  procedures  to  partition  k  normal  popu¬ 
lations  into  “good”  or  “bad”  ones,  respectively,  using  the  nonparametric  empirical  Bayes 
approach.  The  relative  regret  risk  of  a  selection  procedure  is  used  as  a  measure  of  its 
performance.  We  establish  the  asymptotic  optimality  of  the  proposed  empirical  Bayes 
selection  procedures  and  investigate  the  associated  rates  of  convergence.  Under  a  very 
mild  condition,  the  proposed  empirical  Bayes  selection  procedures  are  shown  to  have  rates 
of  convergence  of  order  close  to  0{k~i)  where  k  is  the  number  of  populations  involved 
in  the  selection  problem.  With  further  strong  assumptions,  the  empirical  Bayes  selection 
procedures  have  rates  of  convergence  of  order  0{k  where  1  <  a  <  2  and  r  is  an 

integer  greater  than  2. 
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1.  Introduction 


Consider  k  independent  normal  populations  Xi  =  N{di,a^),  i  =  with  un¬ 

known  means  . . . ,  respectively,  and  a  common  variance  cr^.  Let  9o  denote  a  standard 
or  a  control.  A  population  tt*  is  said  to  be  good  if  di  >  9o,  and  bad  otherwise.  In  certain 
practical  situations,  one  may  be  interested  in  the  selection  of  all  good  populations  while 
excluding  all  bad  populations.  For  example,  let  9i  denote  the  quality  level  of  a  newly 
developed  manufacturing  process  7ri,i  =  and  9o  be  a  specified  standard  level. 

Then,  one  may  be  interested  in  finding  out  all  the  potential  manufacturing  processes  for 
further  investigation.  The  preceding  described  selection  goal  can  also  be  viewed  as  a  first 
step  of  a  selection  problem  in  which  the  selection  goal  is  to  select  the  best  from  among  k 
populations  provided  that  the  best  is  at  least  as  good  as  the  specified  standard  level. 

In  the  literature,  the  problem  of  comparing  normal  populations  with  a  control  has 
been  extensively  studied  by  many  authors.  To  mention  a  few,  for  example,  Dunnett 
(1955),  Gupta  and  Sobel  (1958)  and  Tong  (1969)  have  proposed  and  studied  some  natural 
selection  procedures.  Lehmann  (1961)  and  Spjptvoll  (1972)  have  treated  the  problem  using 
methods  from  the  theory  of  testing  hypothesis.  Randles  and  Hollander  (1971),  Miescke 
(1981)  and  Gupta  and  Miescke  (1985)  have  derived  optimal  procedures  via  minimax  or 
F-minimax  approaches. 

The  purpose  of  this  paper  is  to  derive  statistical  procedures  which  partition  the  k 
normal  populations  into  “good”  and  “bad”  ones,  respectively,  using  the  empirical  Bayes 
approach.  It  is  assumed  that  the  parameter  is  the  value  of  a  specified  standard  level, 
and  therefore  is  assumed  to  be  known. 

The  paper  is  organized  as  follows.  In  Section  2,  the  statistical  model  of  the  selection 
problem  is  introduced  and  a  Bayes  selection  procedure  for  the  selection  problem  is  also 
derived.  As  seen  in  the  later  part  of  the  paper,  the  Bayes  selection  procedure  depends  on 
the  prior  distribution.  When  the  prior  distribution  is  imknown,  the  Bayes  selection  proce¬ 
dure  cannot  be  implemented.  In  such  a  situation,  using  the  empirical  Bayes  approach  and 
by  mimicking  the  behavior  of  the  Bayes  selection  procedure,  we  have  developed  empirical 
Bayes  selection  procedures  in  Section  fi.  The  relative  regret  risk  of  an  empirical  Bayes 
selection  procedure  is  used  as  a  measure  of  the  performance  of  this  empirical  Bayes  selec- 
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tion  procedure.  We  establish  the  asymptotic  optimality  of  the  proposed  empirical  Bayes 
selection  procedures  in  Section  4.  The  rate  of  convergence  of  the  relative. regret  risks  is 
studied  in  Section  5.  Under  a  very  mild  condition,  the  proposed  empirical  Bayes  selec¬ 
tion  procedures  have  rates  of  convergence  of  order  close  to  With  some  further 

assumptions,  the  empirical  Bayes  selection  procedures  have  rates  of  convergence  of  order 
Q(^-Qf(r-i)/[2r+i])  1  <  q;  <  2  and  r  >  2  is  an  integer. 

2.  The  Selection  Problem  and  A  Bayes  Selection  Procedure 

Let  JTii , . . . ,  XjVn  be  a  sample  of  size  m  taken  from  a  normal  population  tTj-  = 

m 

i  =  1, . . . ,  ^.  All  samples  are  assumed  to  be  mutually  independent.  Let  A’j  =  ^  ^  Xij 

i=i 

k  m 

and  when  m  >  2,  let  Wjk  =  X]  E  “  Xif  l[k{m  -  1)].  Note  that  given  9i,Xi  ~ 

2=1  ji=l 

~  1)  Wklcr"^  ~  —  1)),  and  Xi, . . .  ,Xk  and  Wk  are  mutually  inde¬ 

pendent. 

Let  =  {^  =  (^1, . . . ,  ^fc)|  —  CO  <  <  oo,  i  =  1, . . . ,  A:}  be  the  parameter  space,  and 

let  A  =  {q  =  (ai, . . .  ,ak)\ai  =  0,  1,*  =  1, . . . ,  A:}  be  the  action  space.  When  action  a  is 
taken,  it  means  that  population  Xi  is  selected  as  good  if  Oi  =  1,  and  excluded  as  bad  if 
ai  =  0.  For  each  9  Q.  Q,  and  g  €  A,  the  loss  function  L{9,  q)  is  defined  to  be 

ifc 

^  i{9i,ai)  (2.1) 

j=i 

and 

ii9i,ai)  =  0,(^0  -  9i)I{9o  -  9i)  +  {1-  ai){9i  -  9o)I{9i  -  9o),  (2.2) 

where  J(x)  =  1(0)  if  a;  >  0  (otherwise).  Note  that  in  (2.2),  the  first  term  is  the  loss  due  to 
selecting  tt^  as  good  when  9i  <  9q^  and  the  second  term  is  the  loss  due  to  wrongly  excluding 
TTf  as  bad  when  9i>  9o. 

It  is  assumed  that  for  each  i,  the  parameter  9i  is  a  realization  of  a  random  vari¬ 
able  0,-;  and  0i,...,0jb  are  independently  distributed  with  a  common  but  unknown 
prior  distribution  G.  Under  the  preceding  assumption,  Xi^ . . .  ,Xk  and  Wk  are  mu¬ 
tually  independent,  and  Xi,...,Xfc  are  identically  distributed,  having  a  marginal  pdf 
/(®)  =  /  f{A^i^^)dG{9),  where  f(x\9,a‘^)  denotes  the  pdf  of  a  normal  N(9,  ^)  distri¬ 
bution. 
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Let  Xi  =  {Xii,. . .  ,Xim),i  =  1  ...k  and  X  =  (Xi,. . .  ,Xk).  Let  X  be  the  sample 
space  of  X.  A  selection  procedure  8  =  ((5i, . . . ,  8k)  is  defined  to  be  a  mapping  from  the 
sample  space  X  into  the  product  space  [0, 1]*^,  such  that  for  each  i  =  1, . . . ,  k,8i{x)  is  the 
probability  of  selecting  population  Tj  as  a  good  population  when  X  =  x  is  observed.  Let 
V  be  the  class  of  all  selection  procedures.  Also  let  i2(G,  5)  denote  the  Bayes  risk  associated 
with  the  selection  procedure  8.  It  is  assumed  that  £^[|0i|]  <  oo  so  that  the  Bayes  risk 
R(G,  8)  is  finite.  By  Fubini’s  theorem,  a  straightforward  computation  yields  that  the  Bayes 
risk  R{Gj  5)  can  be  expressed  as: 

k 

Ji(G,S)  =  '£  (2.3) 

and 

f  * 

Ri(G,8i)  =  J  [Oq-  <fii{xi)]8i{x)  p  fj  {xj)dx  +  C  (2.4) 

X 

where  v?i(x,)  =  E[Qi\Xi  =  Xj]  :  the  posterior  mean  of  0j  given  Xi  =  x,;  fj(xj):  the 

marginal  joint  pdf  of  Xj,  and  C  =  f(9-  dQ)dG{9).  Thus,  a  Bayes  selection  procedure 

So 

=  (^51,  • .  • ,  8Bk),  which  minimizes  the  Bayes  risks  among  all  selection  procedures  in  V, 
is  clearly  given  by:  For  each  i  =  1,...  ,k  and  x  €  A", 

ri  ii<fiixi)>9o, 

^Bi{x)  =  i  (2.5) 

^  0  otherwise. 

Algebraic  computation  yields  that 

2 

Ifi{xi)  =  Xi  +  ^f^^\xi)/f(xi)  =  ^i(xi),  (2.6) 

where  /(xj)  is  the  marginal  pdf  of  the  sample  mean  Xi  and  f^^\xi)  denotes  its  corre¬ 
sponding  derivative.  That  is,  the  posterior  mean  <pi(xi)  depends  on  Xi  only  through  the 
sample  mean  value  Xi.  From  (2.5)  and  (2.6),  the  i-th  component  Bayes  selection  procedure 
8Bi  depends  on  x  only  through  Xj.  Therefore,  (2.5)  can  be  expressed  as 
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1  if  i’i(xi)  >  00 
0  otherwise. 


1  ifTi(xi)>0 

0  otherwise. 

where  T,(xi)  =  +  {xi  —  9Q)f{xi).  Note  that  rpi{-)  =  . 

. . .  =  Tfc(-)  since  Xi, . . .  ,Xk  axe  identically  distributed. 


(2.5') 

=  and  Ti(-)  = 


One  can  see  that  the  posterior  mean  is  a  continuous  function  in  Xj;  also  i)i{xi)  is 

strictly  increasing  in  Xi  if  the  prior  distribution  G  is  non-degenerate.  Let  Ai  =  {x|V’i(x)  < 
^o}  and  A2  =  {x|V’i(x)  >  ^o}-  Define 


a  = 


(2.7) 


-00  if  A\  =  (f>. 

Note  that  if  A2  ^  (f)  and  Ai  ^  (f>,  then  —00  <  a*  <  00;  and  if  A2  =  ^,  a*  =  00.  In  terms  of 
a*,  the  Bayes  selection  procedure  6b  can  be  written  as: 


6Bi(xi) 


1  if  Xj  >  a* 
0  otherwise. 


(2.5") 


Finally,  the  minimum  Bayes  risk  is: 

k 

R{G,8b)  =  Ri{G,8Bi),  (2.8) 

i=l 

and 

Ri{G,8Bi)  =  J[0o  - 'tpiixi)]8Bi{xi)f(xi)dxi  +  C.  (2.9) 

In  the  following  analysis,  we  consider  those  prior  distributions  G  such  that  lim 

a;»— 00 

fpi{xi)  <  ^0  <  lim  0,(xi).  Hence  Ai  ^  (j)  and  A2  ^  (j>.  Therefore,  —00  <  a*  <  00. 

3.  Empirical  Bayes  Selection  Procedures 

Since  the  prior  distribution  G  is  unknown,  it  is  not  possible  to  implement  the  Bayes 
selection  procedure  8b  for  the  selection  problem  at  hand.  However,  according  to  the  sta¬ 
tistical  model  described  previously,  the  k  components  share  certain  similarity.  Therefore, 
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the  empirical  Bayes  approach  is  employed  to  incorporate  information  from  among  the  k 
populations  to  provide  robust  selection  procedures  for  each  of  the  k  component  selection 
problems. 


The  proposed  empirical  Bayes  selection  procedures  mimick  the  behavior  of  the  Bayes 
selection  procedure  63.  For  this,  the  forms  of  (2.5')  and  (2.5")  provide  important  moti¬ 
vation  for  the  construction  of  the  empirical  Bayes  selection  procedures.  To  construct  the 
empirical  Bayes  selection  procedtures,  first,  we  need  to  have  estimates  for  f(x)  and 


For  an  integer  r  >  2  and  for  each  i  =  0, 1,  let  be  the  class  of  all  Borel-measurable 
bounded  functions  vanishing  outside  the  interval  (0, 1),  such  that  for  ko  €  /CJ, 


and  for  ki  €  /C[, 


y^ko(y)dy 


r  1  if  i  =  0, 

0  if  i  =  1, . . . ,  r  -  1, 
.  Bi  if  j  =  r; 


(3.1) 


'  1 

1 — I 

II 

0 

if  j  =  0,2 

^  B2 

if  j  =  r. 

(3.2) 


We  may  let  S3  be  a  positive  value  such  that  |A:j(j/)|  <  B3  for  all  y  g  (0, 1)  and 
i  =  0, 1.  Also,  let  h  =  h(k)  be  a  decreasing  function  of  k  such  that  h(k)  — >  0  as  Ar  ^  00. 


Define 


(3.3) 


Note  that  for  each  fixed  AT,-  =  x,-,  /ijk(x,)  and  are  consistent  estimators  of 

/(xj)  and  f^^\xi),  respectively;  see  Singh  (1977,  1979). 

When  the  variance  cr^  is  known,  for  each  i  =  1, . . .  ,k,  let 


Also,  let  {CD  be  a  sequence  of  positive  numbers  such  that  is  increasing  in  k  and 
Cl  CO  as  k  oo.  We  consider  an  empirical  Bayes  selection  procedures  =  (5*, . . . , 
defined  as  follows, 

'  0  if  either  {Xi  <  -Cl) 

or  {\Xi\  <  Cl  and  T*k(Xi)  <  0), 

J*(X)=(5,(Xi,X(i))=  (3.4) 

1  if  either  {Xi  >  Cl) 

,  or  (iXil  <  Cl  and  T*k{Xi)  >  0), 
where  X(i)  -  (Xi, . . .  ,Xi-i,Xi+i, . . .  ,Xk). 

When  the  variance  cr^  is  unknown,  we  estimate  by  Wk-  For  each  i  =  let 

fik{Xi)  =  {Xi  -  9o)fik{Xi)  +  —  {Xi). 

jyif 

/S  ^ 

We  then  consider  an  empirical  Bayes  selection  procedure  S  =  {8i, . . .  ,6k)  defined  as  follows, 

'  0  if  either  {Xi  <  —Cl) 

or  {\Xi\  <  Cl  and  fik{Xi)  <  0), 

^i(X)=5i(Wi,.A(0,Wfc)=  {  (3.5) 

1  if  either  {Xi  >  Cl) 

,  or(|Xi|<Qandf’ifc(Xi)>0). 


The  Bayes  risk  of  the  empirical  Bayes  selection  procedure  6*  is: 


R{G,6')  =  Y,Ri(G,6:) 


ana 

Ri{G,Sl)  =El[  j [Bo  -  i^i{xi)]6l{xi,X{i))f{xi)dxi\  +  C 

(3.7) 

=  y [^0  -  Hxi)]El[8l{xi,X{i))]f{xi)dxi  +  C, 
where  the  expectation  E*  is  taken  with  respect  to  the  probability  measure  generated  by 

m- 

The  Bayes  risk  of  the  empirical  Bayes  selection  procedure  8  is: 

k 

'  R{G,8)  =  '^Ri{Gji)  (3.8) 
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and 


Ri{G,8i)  =  j  [9o-i’ii^i)]Hk^i^^{i),WkMxi)dxi  +  C.  (3.9) 

where  the  expectation  Ei  is  taken  with  respect  to  the  probability  measure  generated  by 

m),Wk). 

Since  is  the  Bayes  selection  procedure,  for  any  selection  procedure  6  =  (^i, . . . , 
Ri(G,  6i)  -  Ri{G,  6Bi)  >  0,  i  =  1, . . . ,  A;  and  R{G,  S)  -  R(G,Sb)  >  0.  Define 

p(G,  SJ  =  [R(G,  SJ  -  R(G,  Sb)J/R(G,  Sb).  (3.9) 

p(G,  8)  is  called  as  the  relative  regret  risk  of  the  selection  procedure  8  compared  with 
the  Bayes  selection  procedure  8b.  The  relative  regret  risk  p{G,S)  is  used  to  measure  the 
performance  of  the  selection  procedure  8. 

Definition  3.1.  (a)  A  selection  procedure  8  is  said  to  be  asymptotically  optimal  if 

p(G,  8)  —*  0  as  k  —>■  oo. 

(b)  A  selection  procedure  8  is  said  to  be  asymptotically  optimal  of  order  {0*}  if  p{G,  8)  = 
0(Q!fc)  where  {a^}  is  a  sequence  of  decreasing  positive  numbers  such  that  nfc  ^  0  as 

k  —i.  CO. 

The  asymptotic  optimality  of  the  empirical  Bayes  selection  procedures  8*  and  8  will 
be  investigated  in  the  next  two  sections. 

4.  Asymptotic  Optimality  of  the  Empirical  Bayes  Selection  Procedures 

Under  the  preceding  described  statistical  model  and  the  loss  function,  one  can  see 
that  for  the  Bayes  selection  procedure  ^5,  R\{G.,8bi)  =  ...  =  Rk{G,8Bk)  and  i2(G,^B) 
=  k  R\{G,  8b\). 

Also,  by  the  symmetric  properties  of  the  empirical  Bayes  selection  procedures  8*  and 

5,  we  have  i?i((S',^i*)  =  ,. . .  =  Rk{G,  8%)  axid  R{G,8J')  =  k  Ri{G,  81),  R^{GJi)  =  ...= 
Rk(G,8k)  and  R{G,8)  =  k  Ri{G,Si). 

Therefore,  p{G,8*)  =  [i?i(G,,5i*)  -  Ri{G,8Bi)]/RiiG,8Bi)  and  p{G,8)  =  [i?i(G,.5i) 
— i2i(<j,  ^Bi)]/-Ri((j,  ^si)-  Since  i?i(G,  <fsi)  is  a  fixed  positive  value,  to  study  the  asymp¬ 
totic  optimality  of  the  empirical  Bayes  selection  procedures,  it  suffices  to  investigate  the 
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asymptotic  behavior  of  Ri(G,6*)  —  Ri(G,8bi)  and  Ri{G,Si)  —Ri{G,6bi)  for  sufficiently 
large  k. 

Theorem  4.1  For  the  statistical  model  previously  described  in  Section  2,  assume  that 
<  oo  and  a*  is  a  finite  number.  Then,  the  empirical  Bayes  selection  procedures 
S*  and  8  are  asymptotically  optimal  in  the  sense  that  p(G,8*)  — +  0  and  p(G,8)  0  as 

k  —*  oo. 

Proof:  It  is  assumed  that  k  is  sufficiently  large  so  that  a*  €  {—C^,  C^),  where  {CD  is  a 
sequence  of  increasing  positive  numbers  such  that  lim  =  oo  as  described  in  Section 

k-^oo 

3.  By  the  finiteness  of  a*  and  from  {2.5"),  (2.9),  (3.4)  and  (3.7), 

Ri(G, S',) -R,{G, Sbi) 

=  j  [Oo -■tpi{xi)]ED^t{xi,X(l))  -  8Bi{xi)}f{xi)dxi 

a* 


+  J  [M^i)-0o]P{8*ix^,X{l))==0,8Bi{xx)  =  l}f{xi)dx,. 

a* 

Also,  from  (2.5"),  (2.9),  (3.5)  and  (3.9), 

R^{GJD-MG,8bi) 

a* 

=  j  [Oo-i’i(xi)]P{8i{xi,X(l),Wk)  =  l,8Bi{xi)  =  0}f(xi)dxi 
-Cl  (4.2) 

+  J li^iixi)  -  0o]P{^i(iCi,^(l),  Wjk)  =  0,^5i(a:i)  =  l}f{xi)dxi. 

a* 

From  a  corollary  of  Robbins  (1964),  to  prove  the  asymptotic  optimality  of  8*  and 
it  suffices  to  show  that  for  each  xi,  and  e  =  0, 1, 

P{6*(xi,X(l))  =  e,8Bi{xi)  =  1  -  e}  -4  0 
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^{^:(a:a,X(l),PF,)  =  e,  ^bi(xO  =  1  -  e} 0 

as  -j-  oo.  Note  that  for  each  fixed  Xi,fii.{x-,)  and  (  ^ 

fix,)  and  /(l)(x.).  respectively  AIs  Br  °f 

■  y-  °,  Wi  IS  a  consistent  estimator  of  t7‘‘  Hence  T*  ir  i 

.3  a  consistent  estimator  of  Ti(x.)  for  the  case  and  f  t  i  '  “ 

estimator  of  T,(xi )  for  the  ci^  „„known  case.  Therefore,  ’  “  "  " 

P{^\{xi^X{l))  =  1,6bi{xi)  =  0} 


^{T\k{xi)  >  0,  Ti{xi)  <  0}  ^  0  as  A; 

^K(2^i,^(1))  =  0,<Jsi(xi)  =  1} 


oo,  and 


P{Tik{xi )  <  0,  Ti (xi )  >  0}  0  as  A;  ->  oo. 

Similarly, 

P{M.1,X(1).  =  S.  Mxi)  =  1  -  e} 0  as  h  ^  CO. 

Hence  the  proof  of  the  theorem  is  complete 

C 

5.  Rates  of  Convergence 

The  following  theorem  gives  the  main  rp<?ii)f«  nf  hi, 

*ence  of  the  empirical  Ba.es  selection  p“ 

Theorem  5.1  For  the  statistical  model  described  in  Section  2,  assnme  that  FflO  II 
and  a*  is  a  finite  number.  <  °° 

(a)  If  we  take  h  =.  c,{k  -  l)-i/(2r+i)  ^nd  C*  =  p  h-i  f 

and  cs,  then.  p(..d,  =  0(1-^)  and  0  ^, 

(b)  Furthermore,  suppose  that  for  some  1  <  o  <  2,  and  for  some  h  >  0, 

(A1)|  N»M^J^^^_<^and(A2)  / 

where  A!f(ii,A)  and  Mx,  A)  aredefi  d-  r  f"  '(xi)  x,  <  (x, 

h  -  lu  1-1,1,  x.i  '  '*  If  ™  ‘ake 

Proof  We  provide  pmof ’ofSi'ther  =  Oik'^) 

and  henc  ■  ^ttd  T  ""  -  ‘I^at  of  ^ 

omitted  here.  Also,  assume  that  A  is  large  enough  so  that  a*  e  (-C*  c*)' 

^  k  J ' 
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From  (4.1)  and  Lemma  6.1  -  6.3,  by  noting  that  [i^i(xi)  -  do]f{xi)  =  Ti(xi),  we  have, 


R^{G,S;)-Ri{G,6bi) 


<  j  £(iT,vx,)-Ti(^^ir]/ m(xor'<Jx. 


< 


&i(q) 

P  -  l)/l3]a/2 


/ 


Cl 

+  J  iV“(xi,6)  /  |Ti(xi)r-Vxi 


+ 


62(a) 

[(jfc  -  l)h]“/2 


Cl 

j  In -«„rM»/2(j,j,fc)/|T,(x,)|‘>-i 


dxi 


Cl 

+  h(a)h°‘^  J  \xi-9o\^N°{xi,h)  /  \Ti{x,)\^-^dxi. 

-Cl 

(5.1) 

Since  /(xi)  =  /  e  dG{d)  <  for  all  xi, 

0<M(xi,/i)=  f  f{x\—vh)dv<B4,  (5-2) 


for  all  xi  and  h.  Also, 


d^m 

dv 


xi—hw 


m(x‘i  —  /iiu  — 0 

e  272 


y~]  aj(xi  —  /iu>  —  dy 


‘->=0 


d(?(^) 


where  aj^j  =  0, . . . ,  r,  are  finite  numbers.  Therefore, 


d^m 

d  f 


L  U  =  xi^hwJ 

positive  number  B5  for  all  xi,  w  and  h  and  hence,  for  all  xi  and  h 


<  Bs  for  some 


iV(xi, h)  <  B5. 


(5.3) 
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For  a  =  1,  substituting  the  inequalities  of  (5.2)  and  (5.3)  into  (5.1)  we  obtain, 


<  261(1)5^"  Ct  /  p  -  +26i(1)B5  Ct 

Cl 

+  62(1)5^^  J  \xi -do\dxi  /  [{k-l)hY^'^ 

-Cl 

Cl 

+  b2{l)B5  J  |a:i  —  9o\dxi 

-Cl 

<  Ct  /  p  -  +2bi{l)B5  Ct 

+  |2|«o|CJ  +  /  l(i  -  l)h)‘/^ 

=  o(c;  /  i(k  -  +  0{c;  k-') 


(5.4) 


+  0(Cf  /  [(h  -  l)/i]'/")  +  0(Cf  h'). 

Thus,  if  we  let  h  =  C\{k  —  (7*  =  C2h~^  where  ci  and  C2  are  positive 

constants,  then,  from  (5.4),  i?i(G,^i)—  i2i(G,6Bi)  = 

This  completes  the  proof  of  part  (a). 

For  each  k,  let  1^  =  [— C^,a*  — co)U(a*+co,  Ct]-  Since  Ti(xi)  =  [^i(xi)  — ^o]/(a^i),  hy 
Lemma  6.4,  for  xi  E  J,  |Ti(xi)|  >  |xi  — a*l/(xi)63,  and  for  xi  €  J^,  |ri(xi)|  >/(xi)64. 
Combining  these  inequalities  with  (5.1)  together,  we  obtain 
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where 


^  _ h{a)Ei  bi{a)E2  E3 

-  b^-\{k  -  ^  ^ 

^  bx{a)h°‘(^-^^E^  _ b2(a)E5  b2{a)Ee 

((k  -  l)h)“/2  ■*■  ((jt  _  i)h)°‘/2  5^1 


b2{a)h°‘^E7  b2ia)h°‘^E8 

6r'  ’ 


^  _  f  M“/2(xi,h)  , 

'  7/  ll^l-a-|/(a^i)l“-‘  ■' 


f  M-Z^ixuh) 


iV“(xi,/i) 

[|xi  -q:*|/(xi)]' 


— rC?Xi, 


|xi-0oriV/“/2(:ri,/i), 


^  _  /■|ii-«ol"iV»(i,.A)  . 


-L 


ki-^or  iv«(xi,/i) 

/“-^xi) 


Since  J  is  a  bounded  interval  and  0  <  M(xi,h)  <  S4,  0  <  N{xi,h)  <  B5  for  all 
xi  and  h,  one  can  see  that  for  1  <  a  <  2,  0  <  fJj  <  co,i  =  1,3, 5, 7.  Also,  under  the 
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assumptions  (Al)  and  (A2),  0  <  Ei  <  OQ,i  =  2, 4, 6,8.  Therefore, 

R,{G,SI)-Ri(G,Sbi)  =  0(((k  -  + 

If  we  take  h  ={k  —  then  -  Ri(G,Sbi)  —  ). 

Hence,  this  completes  the  proof  of  part  (b).  □ 

Remarks 

(a)  In  Theorem  5.1(a),  we  have  demonstrated  that  under  the  very  mild  conditions  that 

<  °°  a*  is  finite,  the  relative  regret  risk  converges  to  zero  with  a  rate  of 

r  —  2 

order  0(k  )  where  r  >  2  is  a  positive  integer,  involving  in  the  choice  of  the  kernels 

ko  and  ki,  see  (3.1)  and  (3.2).  According  to  (3.1)  and  (3.2),  there  is  no  restriction 
about  the  choice  of  the  value  r.  Therefore,  we  may  choose  r  as  large  as  possible,  so 
that  the  rate  of  convergence  of  the  relative  regret  risk  may  have  an  order  close  to 
0(k-^/^). 

(b)  Though  under  the  assumptions  Al  and  A2,  the  empirical  Bayes  selection  procedure 
S*  may  have  a  better  convergence  rate,  it  is  possible  that  the  assumptions  Al  and/or 
A2  may  not  hold  for  any  1  <  a  <  2  and  any  0  <  /i  <  1.  For  example,  consider  a  prior 
distribution  with  density  g(&)  where 

fi  if|«l<l. 

m  = 

ifi«i>i. 

Then,  one  can  see  that  f(x)  is  symmetric  about  the  point  0  and  f(x)  is  decreasing 
in  X  for  x  >  0.  Hence,  for  x  >  vh  >  0,  M(x,h)  =  f(x  —  vh)dx  >  f{x).  Also,  as 
X  >  2,  /(x)  >  ^  for  some  c  >  0.  Therefore, 

\x\^  M^^^{x)  x“  M“/2(x,;i)^_ 

7-00  ""-h 

/oo 

x“  f-°‘l^{x)dx 
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x^°‘-^dx 


=  oo  since  1  <  a  <  2. 

Therefore  A1  does  not  hold  in  this  case. 

(c)  We  provide  an  example  in  which  both  the  assumptions  Al  and  A2  hold  for  any  1  < 
a  <  2. 

Assume  that  ©i  ~  iV(0,r^).  Then,  marginally  X\  ~  N{0,q)  where  q=  ^  +r^.  We 
need  to  verify  that 


|a;|“  M“/2(x,h)  ,  ^  f\x\°‘ N°‘{x,h)  ^ 

— h— — <  CO  and  /  h-!_ —  ;  ^ '-dx  <  oo. 


Since  f{x)  =  ^  e 


1 

=  6  2o 


M(x,h)  2vhx-v^h'^ 

- =  /  6  2,  dv  < 


1  if  X  <  0, 


/la: 

e  1  if  X  >  0. 


Hence, 


/ 

/“-‘W 

x\°‘f^  °‘^‘^(x)dx  +  f  x“  (x)e  « 

Jo 

Also, 

=  — - -  6  2,  [  aj(x  —  hwy 

V2’r,  d 

^  t=x-hw 

for  some 

real  values  aj^j  =  0, 1, . . 

. ,  r.  Then, 

a 

5  5  r  1 

2hwx-h^w^  V  ,  T 

f{^) 

=  e  2g  \  aj(x  —  All/;)’'  . 

-  j=zO 

Hence, 

N{x,h)  ^ 

f{^) 

r 

A 1 X 1  *  ' 

e’T”  sup  2.  \^j\  \^‘-hwy. 
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Therefore, 


/  ^  S  J  \x\°‘f{x)\aj\  \x  -  hwy  dx 


<  oo. 


That  is,  both  the  assumptions  Al  and  A2  hold. 


6.  Useful  Lemmas 


The  following  lemmas  are  useful  for  presenting  a  concise  proof  of  Theorem  5.1. 
Lemma  6.1  (a)  For  xi  <  a*  and  a  >  0, 

P{iii*(i,,z(i))  =  1}  < £[|r,*4(i,) - T.(ii)|“]  /  iTiCior. 

(b)  For  Xi  >  a*  and  o;  >  0. 

PW(xi,X(l))  =0}  <B(|Tiyxi)-T,(i,)n  /  |Ti(i,)r. 

Proof:  For  Xi  <  a*,Ti(x)  =  +  (ri  —  ^o)f(^i)  <  0.  Then  by  the  definition  of 

the  empirical  Bayes  selection  procedure  S*  and  by  Markov  inequality, 

p{<ir(xi.^(i))  =  1}  =p{T,\(x,)  >  0} 

=P{^rjfc(3^i)  -  >  -ri(xi)} 

<£[|r.Vi.)-Ti(i,)n/|r.(xi)r. 

Part  (b)  can  also  be  obtained  in  a  similar  way.  Q 

Lemma  6.2  For  0  <  a  <  2,  for  each  fixed  xi, 

^[raxi)-Ti(xi)n 

+  c||x,  -  «„!“{  Var"/2(/u(x,))  +  i£/»(ii)  -  /(x,)|“}, 
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where 


Cc.= 


1  if  0  <  a  <  1, 
2“-i  if  1  <  q;  <  2. 


Proof:  By  the  definitions  of  and  T\{xi)  and  by  Cr-inequality, 

E[\T*,{x^)-T,{x,r\ 


=  E 


^  /u  (®i)  +  (2^1  -  WikM 


2 


2  ^;[i/u(»i)-/''*(^i)r] 


2  \  a 


(6.1) 


(»), 


Again,  by  Cr-inequality,  for  0  <  a  <  2, 

(6.2) 

<  C„£[|/<j[>(xO  -  B/<i>(i.)r]  +  C.IE/fi’)!,)  -  /<«(x.)i“ 


<  C.  +  C„|E/{»(xi)  -  /l'>(x.)r, 

and 

£ii/u(xi)-/(xi)ri 


<  C'.B(|/u(ii)-i;/u(n)n  +  C„|i:/»(ii)-/(ii)r  (6.3) 

<  C,  Var"/'“(/u(xi))  +  C„|£/u(xi)  -  /(x,)|“. 

Substituting  (6.2)  and  (6.3)  into  (6.1)  yields  the  resiilt  of  the  lemma.  □ 

Lemma  6.3  For  each  fixed  xi, 

(a)  Vax  ifikixi))  <  Bl  M{xuh)  /  [{k- l)h], 

(b)  Vari/ll'M)  <  B|M(x,,/»)/[(t-l)/.»), 
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(c)  \Efuixi)-f{xi)\<Bs  N{xi,h), 

(d)  \Ef[l\xi)  -  fi^){xi)\  <  B^h^-^Nixx.h), 

(e)  £J[|lT,(a:i)-ri(xi)r] 


<  h(a){[M(xuh)  /  {(k  -  l)/i3)]“/2  +  [h^-^N{xi,h)]°‘} 


+  b2{a)\xi-eo\°‘{[M{xuh)  /  ((k  -  +  [h^  Nixi,h)]°‘}, 


where  h(a)  =  Cl  62(a)  =  Cl  B^,M{x^,h)  =  /(xi -vh)dv,  and  Nix^^h) 


sup 

0<xy<l 


f  9^m 

d 

t=xi—hwJ 

Proof:  (a)  Var  (/i;t(xi))  =  1-^  Var(fco(^^-^^)) 


ik-l)h^ 


1  c 

=  (rrT)h  I  -  vh)dv 

/  f{xi  —  vh)dv 
Jo 


{k-l)h 


R2 

^  M{xi,h). 


(k-l)h 


Part  (b)  can  be  obtained  in  a  similar  way. 


(c) 


^0  {‘w)f(xi  —  hw)dw. 
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Since  Jp  ko(x)dw  =  1,  hence,  by  the  property  of  fco, 


\Efikixi)  -  f{xi)\  =  \f  ko{w)[f{xi  -  hw)  -  f{xi)]dw\ 

Jo 


ko(w) 


\d^m 

dV 

t=Xi—kw*  - 

{—hwYdw\ 


where  0  <  w*  <  w 


<  Bz  hJ'  sup 

0<u;<l 


\s'm 

[  dV 

=  BzWN{xi,h). 


The  proof  of  part  (d)  can  be  obtained  in  a  way  similar  to  part  (c). 

Finally,  part  (e)  is  obtained  by  plugging  the  results  of  parts  (a)-(d)  into  the  inequality 
in  Lemma  6.2.  □ 


For  a  positive  constant  co,  define  interval  I  =  [a*  —  co,a*  +  cq].  Note  that  tpi{xi)  = 


(1),^  9^e^-'^dG{9)j  [/  9  dG{9)Y] 

mOxi  m9^  ,  9 

[  e  dG(^9) 


m»£i 


f  9  e  20-^  dG(9)  /  f  e  dG{9).  Hence, 

= 

which  is  continuous  in  a^i  and  positive  for  all  xi  since  G  is  nondegenerate. 


Lemma  6.4 

(a)  Let  63  =  inf{?/>i^^(xi)|xi  G  /}.  Then  63  >  0. 

(b)  For  any  xi  €  I,  \fpi{xi)  -  9q\  >  \xi  -  a*\bz. 

(c)  For  any  xi  ^  I,  |V’i(xi)  —  9o  \  >64  for  some  64  >  0. 

Proof:  (a)  Since  V’i^^(xi)  is  continuous  in  xi  on  the  compact  interval  I,  there  exists  an 
X*  €  I  such  that  ^|J[^\x*)  =  63.  Also,  note  that  ip[^\x*)  can  also  be  viewed  as  the 
variance  of  some  non- degenerate  distribution.  Hence  63  >  0. 
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(b)  By  the  definition  of  a*,'^i(a*)  =  Oq.  By  mean- value  theorem,  for  xi  €  /. 

|V’i(a:i)  -  ^o|  -  fpi{a*)\ 

=  l(xi 

>  |xi  -  a*\bz, 

where  is  a  point  between  xi  and  a* . 

(c)  Note  that  tl^\{xi)-dQ  is  strictly  increasing  in  xi  and'!/’i(a*)— =  0.  So,  |V’i(a;i)— ^o|  > 

iidn(|V’i(a*  —  Co)  —  0o\,  \fpi(a*  +  co)  —  ^o|)  =  64  >  0  for  all  xi  ^  I.  □ 
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